Search

The perception of transparent objects for grasp and manipulation remains a major challenge, because existing robotic grasp methods which heavily rely on depth maps are not suitable for transparent objects due to their unique visual properties. These properties lead to gaps and inaccuracies in the depth maps of the transparent objects captured by depth sensors. To address this issue, we propose an end-to-end network for transparent object depth completion that combines the strengths of single-view RGB-D based depth completion and multi-view depth estimation. Moreover, we introduce a depth refinement module based on confidence estimation to fuse predicted depth maps from single-view and multi-view modules, which further refines the restored depth map. The extensive experiments on the ClearPose and TransCG datasets demonstrate that our method achieves superior accuracy and robustness in complex scenarios with significant occlusion compared to the state-of-the-art methods.

Video generation primarily aims to model authentic and customized motion across frames, making understanding and controlling the motion a crucial topic. Most diffusion-based studies on video motion focus on motion customization with training-based paradigms, which, however, demands substantial training resources and necessitates retraining for diverse models. Crucially, these approaches do not explore how video diffusion models encode cross-frame motion information in their features, lacking interpretability and transparency in their effectiveness. To answer this question, this paper introduces a novel perspective to understand, localize, and manipulate motion-aware features in video diffusion models. Through analysis using Principal Component Analysis (PCA), our work discloses that robust motion-aware feature already exists in video diffusion models. We present a new MOtion FeaTure (MOFT) by eliminating content correlation information and filtering motion channels. MOFT provides a distinct set of benefits, including the ability to encode comprehensive motion information with clear interpretability, extraction without the need for training, and generalizability across diverse architectures. Leveraging MOFT, we propose a novel training-free video motion control framework. Our method demonstrates competitive performance in generating natural and faithful motion, providing architecture-agnostic insights and applicability in a variety of downstream tasks., Comment: Project Page: https://xizaoqu.github.io/moft/

The exponential growth of artificial intelligence (AI) and machine learning (ML) applications has necessitated the development of efficient storage solutions for vector and tensor data. This paper presents a novel approach for tensor storage in a Lakehouse architecture using Delta Lake. By adopting the multidimensional array storage strategy from array databases and sparse encoding methods to Delta Lake tables, experiments show that this approach has demonstrated notable improvements in both space and time efficiencies when compared to traditional serialization of tensors. These results provide valuable insights for the development and implementation of optimized vector and tensor storage solutions in data-intensive applications, contributing to the evolution of efficient data management practices in AI and ML domains in cloud-native environments

We advocate for the prioritization of high-precision photometric monitoring of exoplanet and brown dwarf companions to detect brightness variability arising from features in their atmospheres. Measurements of photometric variability provide not only an insight into the physical appearances of these companions, but are also a direct probe of their atmospheric structures and dynamics, and yield valuable estimates of their rotation periods. JWST is uniquely capable of monitoring faint exoplanet companions over their full rotation periods, thanks to its inherent stability and powerful high-contrast coronagraphic imaging modes. Rotation period measurements can be further combined with measurements of v sin i obtained using high-resolution spectroscopy to infer the viewing angle of a companion. Photometric monitoring over multiple rotation periods and at multiple epochs will allow both short- and long-term time evolution in variability signals to be traced. Furthermore, the differences between the layers in a companion's atmosphere can be probed by obtaining simultaneous photometric monitoring at different wavelengths through NIRCam dual-band coronagraphy. Overall, JWST will reach the highest sensitivities to variability to date and enable the light curves of substellar companions to be characterised with unprecedented cadence and precision at the sub-percent level., Comment: 4 pages, 2 figures, white paper submitted in response to the call by the Working Group on Strategic Exoplanet Initiatives with HST and JWST (details at https://outerspace.stsci.edu/display/HPR/Strategic+Exoplanet+Initiatives+with+HST+and+JWST & final report at arXiv:2404.02932), adapted to include author list and affiliations

We present an atmospheric retrieval analysis of HST/WFC3/G141 spectroscopic phase curve observations of two brown dwarfs, WD-0137B and EPIC-2122B, in ultra-short period orbits around white dwarf hosts. These systems are analogous to hot and ultra-hot Jupiter systems, enabling a unique and high-precision comparison to exoplanet systems. We use the PETRA retrieval suite to test various analysis setups, including joint-phase retrievals, multiple temperature structures, and non-uniform abundances. We find that WD-0137B has a dayside that closely resembles that of other ultra-hot Jupiters with inverted temperature structures and H$^-$ opacity, but quickly transitions to a mostly non-inverted temperature structure on the nightside. Meanwhile, EPIC-2122B's atmosphere remains inverted at all constrained longitudes, with dominant H$^-$ opacity. Retrievals with multiple temperature profiles and non-uniform vertical abundances were generally not statistically justified for this dataset, but retrievals with dayside-dilution factors were found to be justified. Retrieving all phases simultaneously with a linear combination of a dayside and nightside atmosphere was found to be an adequate representation of the entire phase-curve once a longitudinal temperature gradient free parameter was included in the retrieval. Comparing to global circulation models, we attribute behavior in the 1D retrievals to the inclined viewing geometry of the systems, which results in always-visible irradiated and inverted portions of the atmosphere "contaminating" spectra measured from the nightside hemisphere. This study sheds light on the similarities between these irradiated brown dwarf systems and hot and ultra-hot Jupiters, but also their unique differences, including the influence of the inclined viewing geometry., Comment: 19 pages, 15 figures, 3 tables. Accepted for publication in ApJ

While imitation learning provides a simple and effective framework for policy learning, acquiring consistent actions during robot execution remains a challenging task. Existing approaches primarily focus on either modifying the action representation at data curation stage or altering the model itself, both of which do not fully address the scalability of consistent action generation. To overcome this limitation, we introduce the Diff-Control policy, which utilizes a diffusion-based model to learn the action representation from a state-space modeling viewpoint. We demonstrate that we can reduce diffusion-based policies' uncertainty by making it stateful through a Bayesian formulation facilitated by ControlNet, leading to improved robustness and success rates. Our experimental results demonstrate the significance of incorporating action statefulness in policy learning, where Diff-Control shows improved performance across various tasks. Specifically, Diff-Control achieves an average success rate of 72% and 84% on stateful and dynamic tasks, respectively. Project page: https://github.com/ir-lab/Diff-Control, Comment: 5 pages, accepted to ICRA 2024 Workshop Back to the Future: Robot Learning Going Probabilistic

Brown dwarfs in ultra-short period orbits around white dwarfs offer a unique opportunity to study the properties of tidally-locked, fast rotating (1-3 hr), and highly-irradiated atmospheres. Here, we present phase-resolved spectrophotometry of the white dwarf-brown dwarf (WD-BD) binary SDSS 1557, which is the fifth WD-BD binary in our six-object sample. Using the Hubble Space Telescope Wide Field Camera 3 Near-infrared G141 instrument, the 1.1 to 1.7 $\mu$m phase curves show rotational modulations with semi-amplitudes of 10.5$\pm$0.1%. We observe a wavelength dependent amplitude, with longer wavelengths producing larger amplitudes, while no wavelength dependent phase shifts were identified. The phase-resolved extracted BD spectra exhibit steep slopes and are nearly featureless. A simple radiative energy redistribution atmospheric model recreates the hemisphere integrated brightness temperatures at three distinct phases and finds evidence for weak redistribution efficiency. Our model also predicts a higher inclination than previously published. We find that SDSS 1557B, the second most irradiated BD in our sample, is likely dominated by clouds on the night side, whereas the featureless day side spectrum is likely dominated by H$^-$ opacity and a temperature inversion, much like the other highly-irradiated BD EPIC2122B., Comment: 19 pages and 11 figures. Accepted to Astrophysical Journal

Few-shot model compression aims to compress a large model into a more compact one with only a tiny training set (even without labels). Block-level pruning has recently emerged as a leading technique in achieving high accuracy and low latency in few-shot CNN compression. But, few-shot compression for Vision Transformers (ViT) remains largely unexplored, which presents a new challenge. In particular, the issue of sparse compression exists in traditional CNN few-shot methods, which can only produce very few compressed models of different model sizes. This paper proposes a novel framework for few-shot ViT compression named DC-ViT. Instead of dropping the entire block, DC-ViT selectively eliminates the attention module while retaining and reusing portions of the MLP module. DC-ViT enables dense compression, which outputs numerous compressed models that densely populate the range of model complexity. DC-ViT outperforms state-of-the-art few-shot compression methods by a significant margin of 10 percentage points, along with lower latency in the compression of ViT and its variants., Comment: Accepted to CVPR 2024. Note: Jianxin Wu is a contributing author for the arXiv version of this paper but is not listed as an author in the CVPR version due to his role as Program Chair

The remarkable efficacy of text-to-image diffusion models has motivated extensive exploration of their potential application in video domains. Zero-shot methods seek to extend image diffusion models to videos without necessitating model training. Recent methods mainly focus on incorporating inter-frame correspondence into attention mechanisms. However, the soft constraint imposed on determining where to attend to valid features can sometimes be insufficient, resulting in temporal inconsistency. In this paper, we introduce FRESCO, intra-frame correspondence alongside inter-frame correspondence to establish a more robust spatial-temporal constraint. This enhancement ensures a more consistent transformation of semantically similar content across frames. Beyond mere attention guidance, our approach involves an explicit update of features to achieve high spatial-temporal consistency with the input video, significantly improving the visual coherence of the resulting translated videos. Extensive experiments demonstrate the effectiveness of our proposed framework in producing high-quality, coherent videos, marking a notable improvement over existing zero-shot methods., Comment: CVPR 24, Code: https://github.com/williamyang1991/FRESCO, Project: https://www.mmlab-ntu.com/project/fresco/

The partial label challenge in Multi-Label Class-Incremental Learning (MLCIL) arises when only the new classes are labeled during training, while past and future labels remain unavailable. This issue leads to a proliferation of false-positive errors due to erroneously high confidence multi-label predictions, exacerbating catastrophic forgetting within the disjoint label space. In this paper, we aim to refine multi-label confidence calibration in MLCIL and propose a Confidence Self-Calibration (CSC) approach. Firstly, for label relationship calibration, we introduce a class-incremental graph convolutional network that bridges the isolated label spaces by constructing learnable, dynamically extended label relationship graph. Then, for confidence calibration, we present a max-entropy regularization for each multi-label increment, facilitating confidence self-calibration through the penalization of over-confident output distributions. Our approach attains new state-of-the-art results in MLCIL tasks on both MS-COCO and PASCAL VOC datasets, with the calibration of label confidences confirmed through our methodology.

Due to privacy or patent concerns, a growing number of large models are released without granting access to their training data, making transferring their knowledge inefficient and problematic. In response, Data-Free Knowledge Distillation (DFKD) methods have emerged as direct solutions. However, simply adopting models derived from DFKD for real-world applications suffers significant performance degradation, due to the discrepancy between teachers' training data and real-world scenarios (student domain). The degradation stems from the portions of teachers' knowledge that are not applicable to the student domain. They are specific to the teacher domain and would undermine students' performance. Hence, selectively transferring teachers' appropriate knowledge becomes the primary challenge in DFKD. In this work, we propose a simple but effective method AuG-KD. It utilizes an uncertainty-guided and sample-specific anchor to align student-domain data with the teacher domain and leverages a generative method to progressively trade off the learning process between OOD knowledge distillation and domain-specific information learning via mixup learning. Extensive experiments in 3 datasets and 8 settings demonstrate the stability and superiority of our approach. Code available at https://github.com/IshiKura-a/AuG-KD ., Comment: Accepted to ICLR 2024

Recent efforts in using 3D Gaussians for scene reconstruction and novel view synthesis can achieve impressive results on curated benchmarks; however, images captured in real life are often blurry. In this work, we analyze the robustness of Gaussian-Splatting-based methods against various image blur, such as motion blur, defocus blur, downscaling blur, \etc. Under these degradations, Gaussian-Splatting-based methods tend to overfit and produce worse results than Neural-Radiance-Field-based methods. To address this issue, we propose Blur Agnostic Gaussian Splatting (BAGS). BAGS introduces additional 2D modeling capacities such that a 3D-consistent and high quality scene can be reconstructed despite image-wise blur. Specifically, we model blur by estimating per-pixel convolution kernels from a Blur Proposal Network (BPN). BPN is designed to consider spatial, color, and depth variations of the scene to maximize modeling capacity. Additionally, BPN also proposes a quality-assessing mask, which indicates regions where blur occur. Finally, we introduce a coarse-to-fine kernel optimization scheme; this optimization scheme is fast and avoids sub-optimal solutions due to a sparse point cloud initialization, which often occurs when we apply Structure-from-Motion on blurry images. We demonstrate that BAGS achieves photorealistic renderings under various challenging blur conditions and imaging geometry, while significantly improving upon existing approaches.

Giant planets grow by accreting gas through circumplanetary disks, but little is known about the timescale and mechanisms involved in the planet assembly process because few accreting protoplanets have been discovered. Recent visible and infrared (IR) imaging revealed a potential accreting protoplanet within the transition disk around the young intermediate-mass Herbig Ae star, AB Aurigae (AB Aur). Additional imaging in H$\alpha$ probed for accretion and found agreement between the line-to-continuum flux ratio of the star and companion, raising the possibility that the emission source could be a compact disk feature seen in scattered starlight. We present new deep Keck/NIRC2 high-contrast imaging of AB Aur to characterize emission in Pa$\beta$, another accretion tracer less subject to extinction. Our narrow band observations reach a 5$\sigma$ contrast of 9.6 mag at 0.6$''$, but we do not detect significant emission at the expected location of the companion, nor from other any other source in the system. Our upper limit on Pa$\beta$ emission suggests that if AB Aur b is a protoplanet, it is not heavily accreting or accretion is stochastic and was weak during the observations., Comment: Accepted for publication in The Astronomical Journal

Quantum Image Processing (QIP) is a field that aims to utilize the benefits of quantum computing for manipulating and analyzing images. However, QIP faces two challenges: the limitation of qubits and the presence of noise in a quantum machine. In this research we propose a novel approach to address the issue of noise in QIP. By training and employing a machine learning model that identifies and corrects the noise in quantum processed images, we can compensate for the noisiness caused by the machine and retrieve a processing result similar to that performed by a classical computer with higher efficiency. The model is trained by learning a dataset consisting of both existing processed images and quantum processed images from open access datasets. This model will be capable of providing us with the confidence level for each pixel and its potential original value. To assess the model's accuracy in compensating for loss and decoherence in QIP, we evaluate it using three metrics: Peak Signal to Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Mean Opinion Score (MOS). Additionally, we discuss the applicability of our model across domains well as its cost effectiveness compared to alternative methods., Comment: 9 pages, 3 figures

With the advent of Web 3.0, the swift advancement of technology confronts an imminent threat from quantum computing. Security protocols safeguarding the integrity of Web 2.0 and Web 3.0 are growing more susceptible to both quantum attacks and sophisticated classical threats. The article introduces our novel long-distance free-space quantum secure direct communication (LF QSDC) as a method to safeguard against security breaches in both quantum and classical contexts. Differing from techniques like quantum key distribution (QKD), LF QSDC surpasses constraints by facilitating encrypted data transmission sans key exchanges, thus diminishing the inherent weaknesses of key-based systems. The distinctiveness of this attribute, coupled with its quantum mechanics base, protects against quantum computer assaults and advanced non-quantum dangers, harmonizing seamlessly with the untrustworthy tenets of the Web 3.0 age. The focus of our study is the technical design and incorporation of LF QSDC into web 3.0 network infrastructures, highlighting its efficacy for extended-range communication. LF QSDC is based on the memory DL04 protocol and enhanced with our novel Quantum-Aware Low-Density Parity Check (LDPC), Pointing, Acquisition, and Tracking (PAT) technologies, and Atmospheric Quantum Correction Algorithm (AQCA). Utilizing this method not only bolsters the security of worldwide Web 3.0 networks but also guarantees their endurance in a time when quantum and sophisticated classical threats exist simultaneously. Consequently, LF QSDC stands out as a robust security solution, well-suited for Web 3.0 systems amidst the constantly evolving digital environment., Comment: 17 pages, 6 figures

Large-scale generative models are shown to be useful for sampling meaningful candidate solutions, yet they often overlook task constraints and user preferences. Their full power is better harnessed when the models are coupled with external verifiers and the final solutions are derived iteratively or progressively according to the verification feedback. In the context of embodied AI, verification often solely involves assessing whether goal conditions specified in the instructions have been met. Nonetheless, for these agents to be seamlessly integrated into daily life, it is crucial to account for a broader range of constraints and preferences beyond bare task success (e.g., a robot should grasp bread with care to avoid significant deformations). However, given the unbounded scope of robot tasks, it is infeasible to construct scripted verifiers akin to those used for explicit-knowledge tasks like the game of Go and theorem proving. This begs the question: when no sound verifier is available, can we use large vision and language models (VLMs), which are approximately omniscient, as scalable Behavior Critics to catch undesirable robot behaviors in videos? To answer this, we first construct a benchmark that contains diverse cases of goal-reaching yet undesirable robot policies. Then, we comprehensively evaluate VLM critics to gain a deeper understanding of their strengths and failure modes. Based on the evaluation, we provide guidelines on how to effectively utilize VLM critiques and showcase a practical way to integrate the feedback into an iterative process of policy refinement. The dataset and codebase are released at: https://guansuns.github.io/pages/vlm-critic.

This paper presents a unique solution to challenges in medical image processing by incorporating an adaptive curve grey wolf optimization (ACGWO) algorithm into neural network backpropagation. Neural networks show potential in medical data but suffer from issues like overfitting and lack of interpretability due to imbalanced and scarce data. Traditional Gray Wolf Optimization (GWO) also has its drawbacks, such as a lack of population diversity and premature convergence. This paper addresses these problems by introducing an adaptive algorithm, enhancing the standard GWO with a sigmoid function. This algorithm was extensively compared to four leading algorithms using six well-known test functions, outperforming them effectively. Moreover, by utilizing the ACGWO, we increase the robustness and generalization of the neural network, resulting in more interpretable predictions. Applied to the publicly accessible Cleveland Heart Disease dataset, our technique surpasses ten other methods, achieving 86.8% accuracy, indicating its potential for efficient heart disease prediction in the clinical setting.

TWA 27B (2M1207b) is the first directly-imaged planetary-mass (MP ~ 5 MJ) companion (Chauvin et al. 2004) and was observed at 0.9--5.3 micron with JWST/NIRSpec (Luhman et al. 2023). To understand the accretion properties of TWA 27B, we search for continuum-subtracted near-infrared helium and hydrogen emission lines and measure their widths and luminosities. We detect the He I triplet at 4.3 sigma and all Paschen-series lines covered by NIRSpec (Pa alpha, Pa beta, Pa gamma, Pa delta) at 4--5 sigma. The three brightest Brackett-series lines (Br alpha, Br beta, Br gamma) as well as Pf gamma and Pf delta are tentative detections at 2--3 sigma. We provide upper limits on the other hydrogen lines, including on H alpha through Hubble Space Telescope archival data. Three lines can be reliably deconvolved to reveal an intrinsic width Delta v = 67+-9 km/s, which is 60% of the surface freefall velocity. The line luminosities seem significantly too high to be due to chromospheric activity. Converting line luminosities to an accretion rate yields Mdot ~ 5e-9 MJ/yr when using scaling relationships for planetary masses, and Mdot ~ 0.1e-9 MJ/yr with extrapolated stellar scalings. Several of these lines represent first detections at an accretor of such low mass. The weak accretion rate implies that formation is likely over. This analysis shows that JWST can be used to measure low line-emitting mass accretion rates onto planetary-mass objects, motivates deeper searches for the mass reservoir feeding TWA 27B, and hints that other young directly-imaged objects might -- hitherto unbeknownst -- also be accreting., Comment: In press at ApJ. Main text: 10 pages, 5 figures, 1 table, 1 equation. Appendix: 12 figures (line profiles). Comments welcome. v2: added one reference. v3: added RV velocity (probable systemic velocity) of primary to Fig. 3b. v4: no change, only added DOI of open-access, published version (2024 March 18)

Diffusion models have achieved remarkable image generation quality surpassing previous generative models. However, a notable limitation of diffusion models, in comparison to GANs, is their difficulty in smoothly interpolating between two image samples, due to their highly unstructured latent space. Such a smooth interpolation is intriguing as it naturally serves as a solution for the image morphing task with many applications. In this work, we present DiffMorpher, the first approach enabling smooth and natural image interpolation using diffusion models. Our key idea is to capture the semantics of the two images by fitting two LoRAs to them respectively, and interpolate between both the LoRA parameters and the latent noises to ensure a smooth semantic transition, where correspondence automatically emerges without the need for annotation. In addition, we propose an attention interpolation and injection technique and a new sampling schedule to further enhance the smoothness between consecutive images. Extensive experiments demonstrate that DiffMorpher achieves starkly better image morphing effects than previous methods across a variety of object categories, bridging a critical functional gap that distinguished diffusion models from GANs.

The unprecedented medium-resolution (R~1500-3500) near- and mid-infrared (1-18um) spectrum provided by JWST for the young (140+/-20Myr) low-mass (12-20MJup) L-T transition (L7) companion VHS1256b gives access to a catalogue of molecular absorptions. In this study, we present a comprehensive analysis of this dataset utilizing a forward modelling approach, applying our Bayesian framework, ForMoSA. We explore five distinct atmospheric models to assess their performance in estimating key atmospheric parameters: Teff, log(g), [M/H], C/O, gamma, fsed, and R. Our findings reveal that each parameter's estimate is significantly influenced by factors such as the wavelength range considered and the model chosen for the fit. This is attributed to systematic errors in the models and their challenges in accurately replicating the complex atmospheric structure of VHS1256b, notably the complexity of its clouds and dust distribution. To propagate the impact of these systematic uncertainties on our atmospheric property estimates, we introduce innovative fitting methodologies based on independent fits performed on different spectral windows. We finally derived a Teff consistent with the spectral type of the target, considering its young age, which is confirmed by our estimate of log(g). Despite the exceptional data quality, attaining robust estimates for chemical abundances [M/H] and C/O, often employed as indicators of formation history, remains challenging. Nevertheless, the pioneering case of JWST's data for VHS1256b has paved the way for future acquisitions of substellar spectra that will be systematically analyzed to directly compare the properties of these objects and correct the systematics in the models., Comment: 32 pages, 16 figures, 6 tables, 2 appendices

Person Re-Identification (ReID) is a challenging problem, focusing on identifying individuals across diverse settings. However, previous ReID methods primarily concentrated on a single domain or modality, such as Clothes-Changing ReID (CC-ReID) and video ReID. Real-world ReID is not constrained by factors like clothes or input types. Recent approaches emphasize on learning semantics through pre-training to enhance ReID performance but are hindered by coarse granularity, on-clothes focus and pre-defined areas. To address these limitations, we propose a Local Semantic Extraction (LSE) module inspired by Interactive Segmentation Models. The LSE module captures fine-grained, biometric, and flexible local semantics, enhancing ReID accuracy. Additionally, we introduce Semantic ReID (SemReID), a pre-training method that leverages LSE to learn effective semantics for seamless transfer across various ReID domains and modalities. Extensive evaluations across nine ReID datasets demonstrates SemReID's robust performance across multiple domains, including clothes-changing ReID, video ReID, unconstrained ReID, and short-term ReID. Our findings highlight the importance of effective semantics in ReID, as SemReID can achieve great performances without domain-specific designs.

In industry, machine anomalous sound detection (ASD) is in great demand. However, collecting enough abnormal samples is difficult due to the high cost, which boosts the rapid development of unsupervised ASD algorithms. Autoencoder (AE) based methods have been widely used for unsupervised ASD, but suffer from problems including 'shortcut', poor anti-noise ability and sub-optimal quality of features. To address these challenges, we propose a new AE-based framework termed AEGM. Specifically, we first insert an auxiliary classifier into AE to enhance ASD in a multi-task learning manner. Then, we design a group-based decoder structure, accompanied by an adaptive loss function, to endow the model with domain-specific knowledge. Results on the DCASE 2021 Task 2 development set show that our methods achieve a relative improvement of 13.11% and 15.20% respectively in average AUC over the official AE and MobileNetV2 across test sets of seven machines., Comment: Submitted to the 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2024)

Synchronous condensers (SCs) play important roles in integrating wind energy into relatively weak power grids. However, the design of SCs usually depends on specific application requirements and may not be adaptive enough to the frequently-changing grid conditions caused by the transition from conventional to renewable power generation. This paper devises a software-defined virtual synchronous condenser (SDViSC) method to address the challenges. Our contributions are fourfold: 1) design of a virtual synchronous condenser (ViSC) to enable full converter wind turbines to provide built-in SC functionalities; 2) engineering SDViSCs to transfer hardware-based ViSC controllers into software services, where a Tustin transformation-based software-defined control algorithm guarantees accurate tracking of fast dynamics under limited communication bandwidth; 3) a software-defined networking-enhanced SDViSC communication scheme to allow enhanced communication reliability and reduced communication bandwidth occupation; and 4) Prototype of SDViSC on our real-time, cyber-in-the-loop digital twin of large-wind-farm in an RTDS environment. Extensive test results validate the excellent performance of SDViSC to support reliable and resilient operations of wind farms under various physical and cyber conditions.

Differentiable Filters, as recursive Bayesian estimators, possess the ability to learn complex dynamics by deriving state transition and measurement models exclusively from data. This data-driven approach eliminates the reliance on explicit analytical models while maintaining the essential algorithmic components of the filtering process. However, the gain mechanism remains non-differentiable, limiting its adaptability to specific task requirements and contextual variations. To address this limitation, this paper introduces an innovative approach called {\alpha}-MDF (Attention-based Multimodal Differentiable Filter). {\alpha}-MDF leverages modern attention mechanisms to learn multimodal latent representations for accurate state estimation in soft robots. By incorporating attention mechanisms, {\alpha}-MDF offers the flexibility to tailor the gain mechanism to the unique nature of the task and context. The effectiveness of {\alpha}-MDF is validated through real-world state estimation tasks on soft robots. Our experimental results demonstrate significant reductions in state estimation errors, consistently surpassing differentiable filter baselines by up to 45% in the domain of soft robotics., Comment: 13 pages, 8 figures, 5 tables, CoRL 2023 workshop Learning for Soft Robots

The Slicer Combined with Array of Lenslets for Exoplanet Spectroscopy (SCALES) instrument is a lenslet-based integral field spectrograph that will operate at 2 to 5 microns, imaging and characterizing colder (and thus older) planets than current high-contrast instruments. Its spatial resolution for distant science targets and/or close-in disks and companions could be improved via interferometric techniques such as sparse aperture masking. We introduce a nascent Python package, NRM-artist, that we use to design several SCALES masks to be non-redundant and to have uniform coverage in Fourier space. We generate high-fidelity mock SCALES data using the scalessim package for SCALES' low spectral resolution modes across its 2 to 5 micron bandpass. We include realistic noise from astrophysical and instrument sources, including Keck adaptive optics and Poisson noise. We inject planet and disk signals into the mock datasets and subsequently recover them to test the performance of SCALES sparse aperture masking and to determine the sensitivity of various mask designs to different science signals.

SCALES (Slicer Combined with Array of Lenslets for Exoplanet Spectroscopy) is a 2 - 5 micron high-contrast lenslet-based integral field spectrograph (IFS) designed to characterize exoplanets and their atmospheres. The SCALES medium-spectral-resolution mode uses a lenslet subarray with a 0.34 x 0.36 arcsecond field of view which allows for exoplanet characterization at increased spectral resolution. We explore the sensitivity limitations of this mode by simulating planet detections in the presence of realistic noise sources. We use the SCALES simulator scalessim to generate high-fidelity mock observations of planets that include speckle noise from their host stars, as well as other atmospheric and instrumental noise effects. We employ both angular and reference differential imaging as methods of disentangling speckle noise from the injected planet signals. These simulations allow us to assess the feasibility of speckle deconvolution for SCALES medium resolution data, and to test whether one approach outperforms another based on planet angular separations and contrasts.

We present aperture masking interferometry (AMI) observations of the star HIP 65426 at $3.8\,\rm{\mu m}$ as a part of the \textit{JWST} Direct Imaging Early Release Science (ERS) program obtained using the Near Infrared Imager and Slitless Spectrograph (NIRISS) instrument. This mode provides access to very small inner working angles (even separations slightly below the Michelson limit of ${}0.5\lambda/D$ for an interferometer), which are inaccessible with the classical inner working angles of the \textit{JWST} coronagraphs. When combined with \textit{JWST}'s unprecedented infrared sensitivity, this mode has the potential to probe a new portion of parameter space across a wide array of astronomical observations. Using this mode, we are able to achieve a contrast of $\Delta m_{F380M}{\sim }7.8$\,mag relative to the host star at a separation of ${\sim}0.07\arcsec$ but detect no additional companions interior to the known companion HIP\,65426\,b. Our observations thus rule out companions more massive than $10{-}12\,\rm{M\textsubscript{Jup}}$ at separations ${\sim}10{-}20\,\rm{au}$ from HIP\,65426, a region out of reach of ground or space-based coronagraphic imaging. These observations confirm that the AMI mode on \textit{JWST} is sensitive to planetary mass companions orbiting at the water frost line, even for more distant stars at $\sim$100\,pc. This result will allow the planning and successful execution of future observations to probe the inner regions of nearby stellar systems, opening essentially unexplored parameter space., Comment: 15 pages, 9 figures, submitted to ApJ Letters

We present a performance analysis for the aperture masking interferometry (AMI) mode on board the James Webb Space Telescope Near Infrared Imager and Slitless Spectrograph (JWST/NIRISS). Thanks to self-calibrating observables, AMI accesses inner working angles down to and even within the classical diffraction limit. The scientific potential of this mode has recently been demonstrated by the Early Release Science (ERS) 1386 program with a deep search for close-in companions in the HIP 65426 exoplanetary system. As part of ERS 1386, we use the same data set to explore the random, static, and calibration errors of NIRISS AMI observables. We compare the observed noise properties and achievable contrast to theoretical predictions. We explore possible sources of calibration errors and show that differences in charge migration between the observations of HIP 65426 and point-spread function calibration stars can account for the achieved contrast curves. Lastly, we use self-calibration tests to demonstrate that with adequate calibration NIRISS F380M AMI can reach contrast levels of $\sim9-10$ mag at $\gtrsim \lambda/D$. These tests lead us to observation planning recommendations and strongly motivate future studies aimed at producing sophisticated calibration strategies taking these systematic effects into account. This will unlock the unprecedented capabilities of JWST/NIRISS AMI, with sensitivity to significantly colder, lower-mass exoplanets than lower-contrast ground-based AMI setups, at orbital separations inaccessible to JWST coronagraphy., Comment: 20 pages, 12 figures, accepted to Astrophysical Journal Letters

Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train generalist X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. More details can be found on the project website https://robotics-transformer-x.github.io., Comment: Project website: https://robotics-transformer-x.github.io

The Slicer Combined with Array of Lenslets for Exoplanet Spectroscopy (SCALES) is a $2-5~\mu$m, high-contrast integral field spectrograph (IFS) currently being built for Keck Observatory. With both low ($R\lesssim250$) and medium ($R\sim3500-7000$) spectral resolution IFS modes, SCALES will detect and characterize significantly colder exoplanets than those accessible with near-infrared ($\sim1-2~\mu$m) high-contrast spectrographs. This will lead to new progress in exoplanet atmospheric studies, including detailed characterization of benchmark systems that will advance the state of the art of atmospheric modeling. SCALES' unique modes, while designed specifically for direct exoplanet characterization, will enable a broader range of novel (exo)planetary observations as well as galactic and extragalactic studies. Here we present the science cases that drive the design of SCALES. We describe an end-to-end instrument simulator that we use to track requirements, and show simulations of expected science yields for each driving science case. We conclude with a discussion of preparations for early science when the instrument sees first light in $\sim2025$., Comment: 10 pages, 16 figures, submitted to Proceedings of the SPIE

Traditional grid analytics are model-based, relying strongly on accurate models of power systems, especially the dynamic models of generators, controllers, loads and other dynamic components. However, acquiring thorough power system models can be impractical in real operation due to inaccessible system parameters and privacy of consumers, which necessitate data-driven dynamic equivalencing of unknown subsystems. Learning reliable dynamic equivalent models for the external systems from SCADA and PMU data, however, is a long-standing intractable problem in power system analysis due to complicated nonlinearity and unforeseeable dynamic modes of power systems. This paper advances a practical application of neural dynamic equivalence (NeuDyE) called Driving Port NeuDyE (DP-NeuDyE), which exploits physics-informed machine learning and neural-ordinary-differential-equations (ODE-NET) to discover a dynamic equivalence of external power grids while preserving its dynamic behaviors after disturbances. The new contributions are threefold: A NeuDyE formulation to enable a continuous-time, data-driven dynamic equivalence of power systems, saving the effort and expense of acquiring inaccessible system; An introduction of a Physics-Informed NeuDyE learning (PI-NeuDyE) to actively control the closed-loop accuracy of NeuDyE; and A DP-NeuDyE to reduce the number of inputs required for the training. We conduct extensive case studies on the NPCC system to validate the generalizability and accuracy of both PI-NeuDyE and DP-NeuDyE, which span a multitude of scenarios, differing in the time required for fault clearance, the specific fault locations, and the limitations of data. Test results have demonstrated the scalability and practicality of NeuDyE, showing its potential to be used in ISO and utility control centers for online transient stability analysis and for planning purposes.

This rapid communication devises a Neural Induction Machine (NeuIM) model, which pilots the use of physics-informed machine learning to enable AI-based electromagnetic transient simulations. The contributions are threefold: (1) a formation of NeuIM to represent the induction machine in phase domain; (2) a physics-informed neural network capable of capturing fast and slow IM dynamics even in the absence of data; and (3) a data-physics-integrated hybrid NeuIM approach which is adaptive to various levels of data availability. Extensive case studies validate the efficacy of NeuIM and in particular, its advantage over purely data-driven approaches.

This letter devises Neural Dynamic Equivalence (NeuDyE), which explores physics-aware machine learning and neural-ordinary-differential-equations (ODE-Net) to discover a dynamic equivalence of external power grids while preserving its dynamic behaviors after disturbances. The contributions are threefold: (1) an ODE-Net-enabled NeuDyE formulation to enable a continuous-time, data-driven dynamic equivalence of power systems; (2) a physics-informed NeuDyE learning method (PI-NeuDyE) to actively control the closed-loop accuracy of NeuDyE without an additional verification module; (3) a physics-guided NeuDyE (PG-NeuDyE) to enhance the method's applicability even in the absence of analytical physics models. Extensive case studies in the NPCC system validate the efficacy of NeuDyE, and, in particular, its capability under various contingencies.